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Mission 

Our  mission  is  to  provide  independent,  relevant,  and  timely  oversight 
of  the  Department  of  Defense  that  supports  the  warfighter;  promotes 
accountability,  integrity,  and  efficiency;  advises  the  Secretary  of 
Defense  and  Congress;  and  informs  the  public. 


Vision 

Our  vision  is  to  be  a  model  oversight  organization  in  the  Federal 
Government  by  leading  change,  speaking  truth,  and  promoting 
excellence — a  diverse  organization,  working  together  as  one 
professional  team,  recognized  as  leaders  in  our  field. 
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Opportunities  (cont'd) 


Objective 

Our  objective  was  to  evaluate  the  sufficiency 
of  Department  of  Defense  [DoD)  mission 
assurance  policies  and  procedures  used  in  the 
acquisition  of  spacecraft  and  strategic  systems. 

Opportunity  for 
Improvement 

Our  evaluation  determined  that  there  were 
no  significant  gaps  or  weaknesses  in  the  DoD 
acquisition  policies  and  procedures  regarding 
mission  assurance.  The  term  "mission  assurance" 
refers  to  the  necessary  systems  engineering, 
design,  quality,  safety,  reliability,  maintainability, 
and  availability  requirements.  Department  of 
Defense  Instruction  [DoDl)  5000.02  "Operation 
of  Defense  Acquisition  Systems"  and  the  Defense 
Acquisition  Guidebook  generally  support  the 
mission  assurance  tenets  through  application 
of  systems  engineering  practices.  However,  the 
Mission  Assurance  Guide  TOR-2007[8546)-6018 
provides  more  detailed  guidance  for  systems 
engineering,  quality  assurance,  and  reliability; 
and  it  should  be  used  by  programs  in  their 
acquisition  process. 

We  found  three  common  program 
management  practices  across  Missile  Defense 
Agency  [MDA),  the  Space  and  Missile  Systems 
Center  [SMC),  and  the  Strategic  Systems 


Program  [SSP)  that  should  be  considered  DoD  standard 
practices.  These  three  practices  are  1)  the  development 
of  specific  policies  and  standards,  which  are  applied  on  every 
program  and  contract,  2)  verifying  program  requirements 
through  in-depth  quality  assurance  audits  of  the  program  and 
contractors;  and  3)  using  independent  organizations  that  report 
directly  to  the  agency  head  to  ensure  mission  success.  These 
practices  help  ensure  a  specific  level  of  mission  success  for 
their  programs. 

Recommendations 

We  recommend  that  the  Deputy  Assistant  Secretary  of  Defense 
for  Systems  Engineering  [DASD[SE)): 

•  Update  the  Defense  Acquisition  Guidebook,  to  recommend 
that  Major  Defense  Acquisition  Programs  [MDAPs)  review, 
tailor,  and  apply  applicable  mission  assurance  concepts  and 
principles,  such  as  those  found  in  the  Mission  Assurance 
Guide  TOR-2007[8546)-6018,  when  developing  Systems 
Engineering  Plans  and  contract  requirements  to  promote  a 
higher  probability  of  mission  success. 

•  Review  the  best  practices  of  Missile  Defense  Agency, 
Air  Force  Space  and  Missile  Systems  Center,  and  Navy 
Strategic  Systems  Program  identified  within  the  report 
and  incorporate  them  into  the  Defense  Acquisition 
Guidebook.  Present  these  practices  at  the  next 
DASD[SE)  bi-monthly  systems  engineering  best  practice 
meeting  to  ensure  dissemination. 
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Management  Comments 

The  Director  of  Acquisition  Resources  and  Analysis  concurred 
with  the  recommendation.  DASD[SE)  will  update  the  Defense 
Acquisition  Guidebook  Chapter  4  by  2015  to  implement  the 
DOD  IG  recommendations  and  will  invite  the  MDA,  SMC 
and  SSP  to  present  their  best  practices  at  a  System 
Engineering  Forum  in  2015. 


DoD  IG  Response 

We  concur  with  the  response.  We  request  to  be  informed 
when  the  Defense  Acquisition  Guidebook  Chapter  4  is 
updated  and  when  MDA,  SMC  and  SSP  are  scheduled  to 
present  at  the  Systems  Engineering  Forum. 
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INSPECTOR  GENERAL 

DEPARTMENT  OF  DEFENSE 

4800  MARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22350-1500 

September  17,  2014 

MEMORANDUM  FOR  PRINCIPAL  DEPUTY  ASSISTANT  SECRETARY  OF  DEFENSE 

FOR  RESEARCH  AND  ENGINEERING 

SUBJECT:  Assurance  Policy  Evaluation  —  Spacecraft  and  Strategic  Systems 
(Report  No.  DoDIG-2014-116) 

We  are  providing  this  report  for  information  and  use.  The  subjeet  evaluation  was  performed  to 
evaluate  the  suffieieney  of  the  Department  of  Defense  (DoD)  mission  assuranee  polieies  and 
proeedure  used  in  the  aequisition  of  spaeeeraft  and  strategie  systems.  Mission  assuranee  is  the  use 
of  industry  best  praetiees  in  systems  engineering,  design,  manufaeturing,  testing,  quality  assuranee, 
risk  management,  reliability,  maintainability,  and  availability  requirements  to  support  overall 
mission  sueeess. 

Our  evaluation  determined  that  there  were  no  signifieant  mission  assuranee  gaps  or  weaknesses 
in  the  DoD  aequisition  polieies  and  proeedures.  However,  we  determined  that  the  Mission 
Assuranee  Guide  TOR-2007(8546)-6018  provides  more  detailed  guidanee  for  system  engineering, 
quality  assuranee,  and  reliability  and  should  be  used  by  programs  in  their  aequisition  proeess. 
Additionally,  we  found  three  eommon  program  management  praetiees  used  by  spaeeeraft  and 
strategie  systems  programs  that  promote  mission  sueeess.  Those  praetiees  are:  1)  The  development 
of  organizational  mission  assuranee  polieies  and  standards,  whieh  are  applied  on  every  aequisition 
program  and  eontraet,  2)  Verifying  program  requirements  through  in-depth  quality  management 
system  audits  of  the  program  and  eontraetors;  and  3)  The  use  of  independent  organizations  reporting 
direetly  to  the  ageney  head  to  ensure  mission  sueeess. 

We  reeommend  that  the  Defense  Aequisition  Guidebook  be  updated  with  the  eoneepts  and 
prineiples  found  in  the  Mission  Assuranee  Guide  TOR-2007(8546)-6018  so  that  Major  Defense 
Aequisition  Programs  ean  ineorporate  best  praetiees  in  their  aequisition  doeuments. 

The  Deputy  Assistant  Seeretary  of  Defense  for  System  Engineering  eoneurred  with  the  findings 
and  reeommendation  in  this  report.  They  stated  the  Deputy  Assistant  Seeretary  of  Defense  for 
System  Engineering  will  update  the  Defense  Aequisition  Guidebook  by  2015  and  will  inelude 
referenees  to  standards  sueh  as  the  systems  engineering  standard  IEEE  15288.1,  the  teehnieal 
reviews  and  audits  standard  IEEE  15288.2,  and  the  eonfiguration  management  standard 
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SAE  EIA-649-1,  once  the  standards  are  published.  Also,  the  Deputy  Assistant  Secretary  of  Defense 
for  System  Engineering  will  invite  the  Missile  Defense  Agency,  the  Space  and  Missile  Systems 
Center,  and  the  Strategic  Systems  Program  to  present  their  best  practices  at  a  Systems  Engineering 
Forum  in  2015. 

We  appreciate  the  courtesies  extended  to  the  staff  Please  direct  questions  to  Captain  Christopher  Failla 
at  (703)  604-8915  (DSN  664-8915),  Christopher.Failla@dodig.mil.  If  you  desire,  we  will  provide  a 
formal  briefing  on  the  results. 


cc: 

Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
Deputy  Assistant  Secretary  of  Defense,  System  Engineering 
Director,  Missile  Defense  Agency 

Commander,  Space  and  Naval  Warfare  System  Command 
Commander,  Space  and  Missile  Systems  Center 
Director,  Navy  Strategic  Systems  Program 
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Introduction 


Objective 

Our  objective  was  to  evaluate  the  sufficiency  of  Department  of  Defense  (DoD) 
mission  assurance  policies  and  procedures  used  in  the  acquisition  of  spacecraft 
and  strategic  systems. 

Background 

The  DoD  space  industry  uses  the  term  "mission  assurance",  which  is  defined  by 
the  Mission  Assurance  Guide  TOR-2007(8546)-6018  (MAG],^  used  by  several 
strategic  space  programs,  as  the  "disciplined  application  of  general  systems 
engineering,  quality,  and  management  principles  towards  the  goal  of  achieving 
mission  success."  DoD  does  not  use  the  term  "mission  assurance"  and  focuses 
on  an  overall  systems  engineering  approach.  We  initiated  this  evaluation  to 
determine  whether  gaps  exist  in  the  overarching  DoD  policy  related  to  systems 
engineering,  manufacturing,  testing,  quality  assurance,  risk  management,  reliability, 
maintainability,  and  availability  requirements  leading  to  mission  success. 

Evaluation  Methodology  and  Criteria 

Methodology 

The  evaluation  was  limited  to  evaluating  overarching  DoD  policy,  and  evaluating 
the  approach  taken  by  several  agencies  to  ensure  mission  success  with  mission 
assurance  principles.  This  evaluation  was  limited  to  DoD  agencies  that  procure 
complex  weapon  systems  that  must  survive  the  harsh  environments  of  space,  such 
as  satellites  and  strategic  missile  systems.  The  selected  agencies  were: 

•  Missile  Defense  Agency  [MDA),  which  is  responsible  for  the 
development  and  operation  of  the  DoD  Ballistic  Missile  Defense  System; 

•  Air  Force  Space  and  Missile  Systems  Center  [SMC),  which  is  responsible 
for  space  programs;  and 

•  Navy  Strategic  Systems  Program  [SSP],  which  is  responsible  for  the 
nuclear  ballistic  missile  program. 


^  The  Mission  Assurance  Guide  TOR-2007(8546)-6081  was  produced  for  U.S.  Government  by  the  Aerospace  Corporation. 
The  primary  purpose  of  the  MAG  is  to  provide  practical  guidance  to  personnel  of  the  Aerospace  Corporation  and  Jn 
general,  National  Security  Space  (NSS)  program  Office  personnel,  who  are  responsible  for  executing  mission  assurance 
functions  that  are  key  to  achieving  program  and  mission  success. 
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We  began  our  evaluation  in  March  of  2013  by  evaluating  DoD  documents 
including  DoD  Instruction  (DoDI)  5000.02  "Operation  of  Defense  Acquisition 
Systems",  the  Defense  Acquisition  Guidebook  [DAG]  May  15,  2013,  and  the 
Systems  Engineering  Plan  (SEP)  outline.  The  team  used  the  2008  version  of 
DoDI  5000.02  for  this  evaluation.  In  November  2013  an  interim  version  of 
DoDI  5000.02  was  released,  stating  that  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  (USD[AT&L]),  with  support  from  the 
Department  of  Defense  Chief  Information  Officer  and  the  Director,  of  Operational 
Test  and  Evaluation  were  to  revise  DoDI  5000.02.  Thus,  we  did  not  use  the 
interim  version  as  it  was  undergoing  revision  at  the  time  of  this  evaluation. 
However,  we  did  evaluate  the  interim  document  and  provided  comment  through 
the  formal  DoD  issuance  process.  We  compared  these  documents  to  the  MAG 
to  determine  if  they  contained  the  tenets  of  a  mission  assurance  program 
described  in  the  MAG.  The  SEP  outline  for  space  programs  specifically  calls 
out  the  MAG,  and  thus  was  used  as  our  mission  assurance  criteria.  We  also 
met  with  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Systems 
Engineering  [ODASD[SE)]  personnel  to  evaluate  their  role  in  implementing 
DoD  policy  and  how  they  use  systems  engineering  to  ensure  mission  assurance 
across  DoD. 

We  then  evaluated  the  selected  agencies,  focusing  on  the  processes  and 
procedures  related  to  design,  manufacturing,  and  quality  assurance  that  support 
the  implementation  of  the  mission  assurance.  At  each  agency,  we  evaluated 
its  documentation  to  include  internal  policies,  procedures,  and  standards  to 
understand  and  evaluate  its  approach  to  mission  assurance.  We  conducted 
interviews  with  agency  system  engineers,  mission  assurance  department  directors, 
and  system  and  quality  engineers  to  determine  how  mission  assurance  practices 
were  implemented.  We  then  analyzed  the  documentation  and  information 
provided  by  engineering  personnel  to  identify  best  practices.  The  detailed 
evaluation  of  each  agency  is  in  Appendix  B  along  with  supporting  analysis,  defining 
commonalities,  and  best  practices. 

DoD  Policies  Related  to  Mission  Assurance 

The  team  compared  three  acquisition  documents;  (1)  DoDI  5000.02,  the 
overarching  acquisition  document;  [2]  the  DoD  guidance  for  developing  a  SEP, 
which  is  a  deliverable  of  DoDI  5000.02;  and  (3)  the  DAG,  a  supporting  guidebook 
against  the  MAG  to  determine  how  mission  assurance  principles  are  incorporated 
into  the  acquisition  process.  The  DoDI  5000.02  governs  the  DoD  acquisition 
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process  and  establishes  the  framework  for  translating  capability  needs  and 
technology  into  weapon  system  acquisition  programs  that  meet  statutory 
requirements.  DoDI  5000.02  requires  program  managers  develop  a  SEP,  which 
outlines  how  the  program  will  meet  its  engineering  requirements.  The  SEP  format 
states  that  programs  operating  under  space  system  acquisition  procedures  describe 
how  their  mission  assurance  processes  meet  the  best  practices  described  in  the 
MAG.  DoDI  5000.02  also  refers  program  managers  to  the  DAG,  which  provides 
program  managers  best  practices  that  can  be  applied  throughout  the  acquisition 
process  to  help  satisfy  its  requirements.  The  MAG  is  a  collection  of  industry 
design,  manufacturing,  quality,  and  safety  best  practices  whereas  the  DAG  is  a 
collection  of  acquisition  life  cycle  best  practices. 

DoDI  5000.02  Operations  of  Defense  Acquisition 

DoDI  5000.02  is  written  by  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  [USD[AT&L)],  with  support  from  the  Department 
of  Defense  Chief  Information  Officer  and  the  Director,  of  Operational  Test 
and  Evaluation  to  identify  the  program  management  functions  and  processes 
necessary  to  acquire  any  system  or  weapon  system.  It  has  gone  through  several 
iterations  to  reflect  priorities  and  evolving  acquisition  policies.  For  example,  early 
versions  emphasized  reviews,  quality  control,  and  design-to-cost.  In  1996,  the 
DoDI  5000.02  was  revised  to  meet  the  1994  Federal  Acquisition  Streamlining 
Act  [FASA],  which  encouraged  the  simplification  of  Government  procedures  to 
procure  items.  The  1996  version  separated  mandatory  policies  and  procedures 
from  discretionary  practices  supporting  the  implementation  of  acquisition 
policy;  these  policies  and  procedures  were  placed  in  DoD  Regulation  5000. 2-R 
"Mandatory  Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and 
Major  Automated  Information  System  (MAIS]"  and  Defense  Acquisition  Deskbook 
respectively.  The  1996  revision  stated  that  by  reducing  mandatory  guidance, 
program  managers  were  free  to  exercise  their  own  judgment  while  managing 
an  acquisition  program.  The  revised  DoDI  reduced  the  burden  of  mandatory 
procedures  and  specifications,  encouraged  prudent  risk  management,  and 
allowed  the  integration  of  commercial  products  and  best  practices.  This  resulted 
in  the  acquisition  process  focusing  on  mission  oriented  program  management 
and  performance  based  contracting.  In  2000,  the  DoDI  5000.02  canceled  and 
replaced  DoD  5000. 2-R. 

The  DoDI  5000.02  (2008)  lists  applicable  laws,  policies,  and  reference 

documents  related  to  the  various  phases  of  the  acquisition  process.  Its  purpose 
is  to  establish  policy  for  the  management  of  all  acquisition  programs  and  the 
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acquisition  process  itself.  It  does  not  provide  technical  program  or  system 
requirements.  However,  DoDI  5000.02  does  include  a  systems  engineering 
enclosure  that  describes  the  policies  and  procedures  regarding  the  application  of 
systems  engineering  to  the  acquisition  process.  The  systems  engineering  section 
briefly  outlines  areas  such  as  risk  management,  technical  reviews,  manufacturing 
and  producibilty,  and  reliability  and  maintainability,  which  a  program  manager 
must  discuss  within  the  SEP. 

Systems  Engineering  Plan  Outline 

DoDI  5000.02  requires  program  managers  to  develop  a  SEP  showing  how  they 
will  meet  systems  engineering  requirements.  The  SEP  is  used  to  describe  the 
programs  overall  technical  approach  to  risk  management,  program  processes, 
resources,  organization,  metrics,  design  considerations  and  the  criteria  for 
technical  reviews.  ODASD(SE]  located  within  the  Office  of  the  Assistant  Secretary  of 
Defense  for  Research  and  Engineering  is  responsible  for  reviewing  and  approving 
all  MDAPs  and  MAIS  SEPs.  On  April  20,  2011,  the  Principal  Deputy  Under 
Secretary  of  Defense  Acquisition  Technology,  and  Logistics  issued  a  memorandum, 
"Document  Streamlining  -  Program  Strategies  and  Systems  Engineering  Plan," 
directing  programs  to  develop  Systems  Engineering  Plans  using  the  approved  SEP 
outline  to  ensure  proper  documentation  of  required  information.  The  SEP  outline 
facilitates  uniformity  of  program  data  submitted  to  DASD[SE]  for  evaluation  and 
approval.  It  also  ensures  programs  are  submitting  all  required  data  to  comply 
with  section  139b  title  10  United  States  Code  (10  U.S.C.  §139b]  and  DoDI  5000.02. 

The  SEP  outline  is  composed  of  four  sections:  Introduction;  Technical  Requirements; 
Engineering  Resources  and  Management;  and  Technical  Activities  and  Products. 
The  Technical  Requirements  section  identifies  the  system  architecture  and  required 
certification,  such  as  airworthiness.  The  Engineering  Resources  Management  section 
focuses  on  schedule,  tasks,  personnel  roles  and  responsibilities,  internal  processes 
such  as  risk  management,  and  the  overall  organizational  structure  of  the  program. 
The  Technical  Activities  and  Products  section  outlines  the  systems  engineering 
activities  of  the  program  and  how  top-level  performance  requirements  (how  fast, 
how  far,  how  big)  are  incorporated  into  the  configuration.  It  also  identifies  the 
technical  review  entrance  and  exit  requirements  for  all  major  reviews,  such  as 
milestone  decisions,  preliminary  design  reviews  (PDR),  and  critical  design 

reviews  (CDR).  In  addition,  the  section  requires  the  program  managers  to  identify 
how  affordability;  corrosion;  environmental,  safety  and  occupational 

health  (ESOH);  Human  System  Integration  (HSI);  Item  Unique  Identification  (lUID); 
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manufacturing,  system  architecture;  program  protection;  and  reliability  and 
maintainability  are  identified  and  incorporated  in  the  contract.  However,  if  the 
program  is  space  based  the  program  manager  would  follow  Table  4.6-1  Design 
Considerations  footnote  3  in  the  SEP  outline,  for  reliability  and  maintainability. 
The  footnote  states, 

"Programs  operating  under  Space  Systems  Acquisition 
Procedures  shall  address  Mission  Assurance  [MA]  planning 
in  the  context  of  reliability  and  provide  a  description  of 
MA  activities  undertaken  to  ensure  that  the  system  will 
operate  properly  once  launched  into  orbit.  Specifically, 
space  programs  will  describe  how  the  Mission  Assurance 
process  employed  meets  the  best  practices  described  in  the 
Mission  Assurance  Guide  (reference  Aerospace  Corporation 
TOR-2007(8547)-6018).  This  description  should  include 
program  phase-dependent  processes  and  planning  for  MA 
in  the  next  phase  of  the  program  and  the  way  program 
MA  processes  adhere  to  applicable  policies  and  guidance. 

Also  describe  the  launch  and  operations  readiness  process." 

The  SEP  demonstrates  how  systems  engineering  principles  are  being  translated 
into  the  program’s  acquisition  process,  thus,  the  SEP  is  the  criteria  on  which  the 
technical  aspects  such  as  manufacturing,  reliability  and  maintainability  are  judged. 
DASD(SE)  must  review  and  approve  the  SEP  before  the  program  proceeds.  Finally, 
DASD(SE)  uses  the  SEP  during  program  reviews  to  ensure  the  programs  are  on 
track  and  engineering  risks  are  being  properly  identified  and  mitigated. 

Defense  Acquisition  Guidebook 

The  DAG,  formerly  the  Defense  Acquisition  Deskbook,  is  designed  to  help  a 
program  manager  meet  the  requirements  outlined  in  DoDI  5000.02.  The  DAG 
is  not  a  requirements  document  and  should  not  be  used  as  such.  It  contains 
non-mandatory  expectations  for  satisfying  the  requirements  of  DoDI  5000.02. 
The  DAG  complements  DoDI  5000.02  by  providing  discretionary  best  practices 
that  can  be  tailored  to  program  needs.  The  program  managers  use  the  DAG  as 
a  reference  to  support  their  decisions  as  well  as  help  them  understand  the 
overall  acquisition  process. 

The  Defense  Acquisition  University  provides  the  DAG  to  the  acquisition 
community  as  an  interactive  website  with  14  chapters  that  align  to  DoDI  5000.02 
requirements.  Each  chapter  lists  potential  ways  the  program  manager  can  satisfy 
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process  and  requirements.  The  DAG  does  not  contain  a  specific  chapter  or 
section  dedicated  to  mission  assurance;  however,  it  mentions  the  principles 
throughout  the  document  specifically  within  Chapter  4,  "Systems  Engineering."  At 
the  time  of  this  report,  the  DAG  was  not  updated  to  align  with  the  current 
2013  revision  of  DoDI  5000.02. 

Mission  Assurance  Guide  TOR-2007(8546]-6018 

The  MAG  is  an  industry  document,  primarily  used  by  the  DoD  space  community  that 
identifies  the  tenets  of  mission  assurance.  The  MAG  defines  mission  assurance  as 
the  disciplined  application  of  proven  scientific,  engineering,  quality,  and  program 
management  principles  towards  the  goal  of  achieving  mission  success,  follows  a 
general  systems  engineering  framework,  uses  risk  management,  and  independent 
assessment  throughout  the  process.  The  Aerospace  Corporation  in  concert  with 
National  Security  Space  [NSS]  community  and  Government  sponsors  developed 
the  MAG  to: 

•  decrease  the  number  of  system  integration  anomalies  and  failures; 

•  prevent  the  weakening  of  systems  engineering  and  mission  assurance 
practices; 

•  reestablish  high  levels  of  mission  success  for  the  NSS  activities;  and 

•  re-invigorate  and  apply  the  principles  and  best  practices  of  mission 
assurance  in  a  formal  and  disciplined  manner  throughout  the  space 
acquisition  process. 

The  MAG’s  primary  purpose  is  to  provide  practical  guidance  for  executing  mission 
assurance  functions  that  are  key  in  achieving  program  and  mission  success. 

The  MAG  describes  the  overarching  mission  assurance  framework,  processes, 
disciplines,  tasks,  best  practices,  standards,  and  procedures  applicable  to  NSS 
programs  to  ensure  mission  success.  The  document  first  identifies  the  mission 
assurance  guiding  principles.  It  explains  how  to  tailor  the  document  to  suit  the 
program’s  needs  and  discusses  mission  assurance  implementation  and  evaluation 
methods.  Finally,  the  document  describes  in  detail  the  tenets  of  mission  assurance, 
which  include  program  assurance,  requirements  developments,  design  assurance, 
manufacturing,  integration,  operations,  reviews  and  audits,  risk  management, 
reliability,  configuration,  parts  and  material,  quality,  safety,  software,  and 
information  assurance. 
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Opportunity  for  Improvement  A 

Mission  Assurance  Guide  Provides  Detailed  Guidance 

We  determined  there  were  no  significant  gaps  or  weaknesses  in  the  DoD 
acquisition  policies  and  procedures  regarding  mission  assurance.  DoDI  5000.02 
and  the  DAG  generally  support  the  mission  assurance  tenets  through  application 
of  systems  engineering  practices.  However,  the  Mission  Assurance  Guide  provides 
more  detailed  guidance  for  systems  engineering,  quality  assurance,  and  reliability 
considerations  supporting  system  acquisition. 


DAG  to  MAG 

The  DAG  is  designed  to  improve  a  program  manager’s  understanding  of  the 
acquisition  process  and  the  statutory  and  regulatory  requirements  associated  with 
the  process  and  guide  them  in  meeting  the  requirements  of  DoDl  5000.02  using 
best  practices.  Conversely,  the  MAG  provides  details  on  how  to  ensure  mission 
success  from  a  product  and  engineering  standpoint  and  contains  activities  specific 
to  space  systems.  Therefore,  when  the  DAG  is  compared  to  the  MAG,  there  are 
differences  in  the  level  of  detail  required  for  space  based  design  assurance; 
manufacturing  assurance;  integration,  test,  and  evaluation;  operations  readiness 
assurance;  reviews  and  audits;  risk  management;  and  reliability  engineering. 

The  DAG  discusses  several  mission  assurance  principles,  in  relation  to  the 
acquisition  process,  but  does  not  offer  the  same  depth  of  information  as  the  MAG. 
The  MAG  is  more  specific  in  the  areas  of  reviews  and  audits,  quality,  and  reliability. 
Overall,  the  DAG  focuses  on  acquisition  programmatics  such  as  cost  and  schedule, 
while  the  MAG  focuses  on  the  technical  engineering  aspects  of  the  program. 

The  DAG  addresses  quality  throughout  the  document  as  it  relates  to  the 
acquisition  process,  it  does  not  contain  a  dedicated  section  to  address  product 
quality  assurance.  However,  the  DAG  does  identify  AS9100  "Quality  Management 
Systems  -  Requirements  for  Aviation,  Space  and  Defense  Organizations"  and 
ISO  9001:2008  "Quality  Management  Systems  -  Requirements"  as  quality 
requirements  to  be  considered  on  any  contract.  In  comparison,  the  MAG  dedicates 
a  chapter  to  quality  assurance,  which  goes  beyond  stating  quality  requirements 
considerations.  The  MAG  also  defines  quality  assurance  and  clearly  outlines 
the  objectives  and  activities  of  both  the  contractor  and  program  office  for 
implementing  a  quality  assurance  program. 
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The  DAG,  Chapter  4,  "Systems  Engineering,”  discusses  reliability  in  relation  to 
the  acquisition  process,  but  not  in  detail.  The  DAG  outlines  considerations  for 
the  contract  and  statement  of  work,  lists  tools  to  calculate  reliability  and  directs 
program  managers  to  additional  reliability  resources.  In  comparison,  the  MAG 
dedicates  a  chapter  to  reliability.  The  MAG  defines  reliability,  identifies  the 
key  practices,  and  describes  the  core  reliability  activities.  The  MAG  also  covers 
worst-case  and  parts  stress  analysis.  The  MAG  also  discusses  critical  and  limited 
life  item  control,  parts  reliability  analysis  and  environmental  stress  screening. 

Recommendation,  Management  Comments,  and 
Our  Response 

Recommendation  A 

We  recommend  DASD(SE)  update  the  DAG  to  recommend  that  MDAPs  review,  tailor, 
and  apply  applicable  mission  assurance  concepts  and  principles,  such  as  those 
found  in  the  MAG,  when  developing  SEPs  and  contract  requirements  to  promote 
a  higher  probability  of  mission  success. 

Principal  Deputy  for  the  Assistant  Secretary  of  Defense 
for  Research  &  Engineering  Comments 

The  Director  of  Acquisition  Resources  and  Analysis  concurred  with  the 
recommendation.  DASD(SE)  will  update  DAG  Chapter  4  to  implement  the  DoD  IG 
recommendations  by  2015. 

DoD  IG  Response 

The  DoD  IG  found  the  comments  responsive,  and  requests  to  be  notified  when  the 
DAG  Chapter  4  is  updated  and  released. 
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Opportunity  for  Improvement  B 

Common  Program  Management  Practices 

We  found  three  common  program  management  practices  across  MDA,  SMC,  and 
SSP  that  should  be  considered  DoD  standard  practices.  These  three  practices  are 
1]  the  development  of  specific  policies  and  standards,  which  are  applied  on 
every  program  and  contract,  2)  verification  of  the  program  requirements  through 
in-depth  quality  assurance  audits  of  the  program  and  contractors,  and  3]  the  use  of 
independent  organizations  reporting  directly  to  the  agency  heads  to  ensure  mission 
success.  These  practices  help  the  program  ensure  they  are  maintaining  a  specific 
level  of  mission  success  for  all  their  space  programs. 


Specific  Policies  and  Standards 

MDA,  SMC,  and  SSP  all  developed  their  own  specific  policies  and  standards,  which 
are  required  on  each  program  to  ensure  quality  and  mission  success.  For  example, 
MDA  uses  the  MDA  Assurance  Provision  (MAP]  and  Parts,  Materials,  and  Processes 
Mission  Assurance  Plan  (PMAP);  SMC  uses  a  list  of  69  standards;  and  SSP  uses 
the  Technical  Program  Management  Requirements  for  Strategic  System  Programs 
Acquisitions  Document  (T9001B],  which  identifies  the  contract  requirements 
that  help  ensure  the  desired  level  of  reliability  and  mission  success.  Each  of  these 
documents  identifies  the  specific  Government  or  industry  standards  that  will 
become  executable  requirements  for  the  contractor.  These  standards  and  policies 
cover  areas  such  as  systems  engineering,  including  design  and  integration  of 
systems,  quality  assurance  and  technical  reviews  and  assessments.  The  application 
of  uniform  standards  allows  the  program  to  maintain  desired  levels  of  mission 
success  and  provide  a  baseline  to  audit  or  evaluate  the  program  to  determine  its 
overall  probability  of  mission  success. 

Technical  Assessments 

MDA,  SMC  and  SSP  all  conduct  in-depth  independent  technical  and  quality 
assessments  of  their  programs.  Each  component  performs  verification  of  its 
program  requirements  by  conducting  technical  and  quality  assurance  assessments 
of  its  contractors.  For  example,  MDA  has  an  audit  program  consisting  of 
three  types  of  audits  that  evaluate  field  activities,  contractors,  and  suppliers; 
SMC  employs  technical  reviews  and  audits  throughout  the  program  lifecycle  as 
milestone  decision  points;  and  SSP  has  five  main  technical  and  quality  reviews 


DODIG-2014-116  |  9 


Opportunity  for  Improvement  B 


that  evaluate  contractors,  field  activities  and  program  offices.  The  assessments 
conducted  by  each  of  these  components  focus  on  the  program’s  adherence  to 
internal  mission  assurance  policies  and  or  the  contractor’s  adherence  to  mission 
assurance,  quality,  and  reliability  standards  as  well  as  the  contractor’s  own  internal 
policies  and  procedures.  These  assessments  include  an  in-depth  assessment  of  the 
products,  management,  design,  inspection,  manufacturing  and  test  processes.  These 
assessment  help  the  components  identify  potential  program  contractor  or  supplier 
practices  that  may  impact  mission  success  and  verify  contractual  requirements  are 
being  met. 

Independent  Organizations 

MDA,  SMC  and  SSP  all  use  an  independent  organization,  reporting  directly  to 
leadership,  to  ensure  mission  success.  For  example,  MDA’s  independent  organization 
is  Quality,  Safety,  and  Mission  Assurance  (MDA/QS]  which  is  responsible  for  the 
agency’s  mission  assurance  strategy;  SMC’s  independent  organization  is  the 
Engineering  Directorate  (SMC/EN)  which  is  responsible  for  the  independent 
assessment  and  analysis  of  programs;  and  SSP’s  independent  organization  is  the 
Office  of  the  Chief  Engineer  (SP201],  under  the  Technical  Division  (SP20],  which 
is  responsible  for  ensuring  that  technical  disciplines  such  as  quality,  reliability, 
maintainability  and  product  assurance  are  included  in  program  activities.  These 
independent  organizations  have  direct  reporting  lines  to  the  agency’s  director. 
In  each  case,  the  independent  organization  must  approve  the  tailoring  of  the 
baseline  policies  and  requirements  by  the  program  office  for  inclusion  on  the 
contract.  Additionally  these  organizations  ensure  compliance  with  the  standards 
and  policies  through  audits  and  direct  engineering  support  to  the  program  office. 
These  organizations  also  participate  in  design  reviews,  material  review  boards, 
and  tests  providing  independent  risk  assessments  to  the  program  manager  and 
component  heads. 

Recommendation,  Management  Comments,  and 
Our  Response 

Recommendation  B 

We  recommend  DASD(SE)  incorporate  into  the  Defense  Acquisition  Guidebook 
the  best  practices  of  MDA,  SMC  and  SSP  that  were  identified  and  highlighted  within 
the  report.  Present  these  practices  at  the  next  DASD(SE)  bi-monthly  systems 
engineering  best  practice  meeting,  to  ensure  dissemination  of  these  best  practices. 
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Principal  Deputy  for  the  Assistant  Secretary  of  Defense 
for  Research  &  Engineering  Comments 

The  Director  of  Acquisition  Resources  and  Analysis  concurred  with  our 
recommendation  stating  the  principles  of  DAG  Chapter  4  are  consistent  with  the 
DoD  IG’s  recommendation.  They  also  stated  the  DAG  will  be  updated  as  new 
practices  emerge  and  will  include  references  to  standards  such  as  the  systems 
engineering  standard  IEEE  15288.1,  the  technical  reviews  and  audits  standard 
IEEE  15288.2,  and  the  configuration  management  standard  SAE  EIA-649-1  once  the 
standards  are  published  and  adopted  by  the  DoD.  Also,  DASD(SE)  will  invite  MDA, 
SMC,  and  SSP  to  present  their  best  practices  at  a  Systems  Engineering  Forum 
in  2015. 

DoD  IG  Response 

The  DoD  IG  found  the  comments  responsive.  We  requests  to  be  notified  when  the  cited 
standards  are  released  and  when  MDA,  SMC  and  SSP  are  scheduled  to  present  at  the 
Systems  Engineering  Forum. 
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Appendix  A 

Scope  and  Methodology 

We  conducted  this  technical  evaluation  from  March  2013  through  June  2014  in 
accordance  with  the  Council  of  the  Inspectors  General  on  Integrity  and  Efficiency, 
"Quality  Standards  for  Inspection  and  Evaluation,"  January  2012.  We  planned 
and  performed  the  evaluation  to  obtain  sufficient  and  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  observations  and  conclusions,  based  on  our 
evaluation  objectives 

We  evaluated  DoD  documents  including  DoD  Instruction  (DoDIJ  5000.02  "Operation 
of  Defense  Acquisition  Systems"  December  8,  2008,  the  Defense  Acquisition 
Guidebook  (DAG)  May  15,  2013,  and  the  Systems  Engineering  Plan  (SEP)  outline. 
We  then  compared  these  documents  to  the  MAG  to  determine  if  they  contain 
the  tenets  of  a  mission  assurance  program  as  described  by  the  MAG.  We  met 
with  the  personnel  from  the  Office  of  the  Deputy  Assistant  Secretary  of  Defense 
for  Systems  Engineering  (ODASD(SE))  to  evaluate  their  role  in  implementing 
DoD  policy  and  how  they  use  systems  engineering  to  ensure  mission  assurance 
across  DoD. 

We  then  evaluated  the  Missile  Defense  Agency  (MDA),  Air  Force  Space  and  Missile 
Systems  Center  (SMC),  and  Navy  Strategic  Systems  Program  (SSP).  At  each  agency, 
we  evaluated  its  documentation  to  include  internal  policies,  procedures,  and 
standards  to  understand  and  evaluate  its  approach  to  mission  assurance.  We 
conducted  interviews  of  agency  directors  and  system  and  quality  engineers  to 
determine  how  mission  assurance  practices  were  implemented.  We  then  analyzed 
the  documentation  and  information  provided  by  engineering  personnel  to  identify 
best  practices. 

Use  of  Computer-Processed  Data 

We  did  not  use  computer-processed  data  to  perform  this  audit. 
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Appendix  B 

Service  and  Agency  Reports 

Missile  Defense  Agency 

Background 

The  Missile  Defense  Agency,  formerly  known  as  The  Strategic  Defense  Initiative 
Organization  [SDIO),  was  established  during  the  Reagan  presidency  in  1983 
to  develop  non-nuclear  missile  defenses.  In  1999  in  accordance  with  Public 
Law  106-38,  "The  National  Missile  Defense  Act,"  SDIO’s  mission  was  to  develop 
and  deploy  an  effective  National  Missile  Defense  System  capable  of  defending  the 
United  States  against  limited  ballistic  missile  attack.  The  mission  was  updated 
under  President  George  W.  Bush  to  develop  an  integrated,  layered  defense  that 
would  be  capable  of  attacking  warheads  and  missiles  in  all  phases  of  their 
flight.  This  is  what  is  known  today  as  Ballistic  Missile  Defense  System  (BMDS). 
The  BMDS  is  comprised  of  multiple  interoperable  subsystems  with  a  mission 
to  intercept  ballistic  missile  threats  in  all  phases  of  flight  as  seen  in  Figure  B-1. 
MDA  manages  and  develops  the  BMDS  with  an  average  budget  of  about  8  billion 
dollars  based  on  fiscal  data  from  FY  2011  through  FY  2014.  To  date,  MDA’s  test 
program  has  had  97  out  of  111  successful  flight  tests  across  their  multiple  systems. 


Figure  B-1:  Ballistic  Missile  Defense  System 
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The  organizational  structure  of  MDA  includes  functional  managers,  program 
managers,  knowledge  center  managers,  and  two  national  teams  (Figure  B-2). 
The  functional  managers  are  comprised  of  deputies  for  operations,  engineering, 
acquisition  management,  advanced  technology,  test/integration  and  fielding,  and 
international  affairs.  Program  Managers  focus  on  executing  each  BMDS  element. 


Figure  B-2.  MDA  Organizational  Chart 
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Mission  Assurance  Approach 

MDA’s  organizational  structure  designates  a  Quality,  Safety,  and  Mission  Assurance 
directorate  (QS),  which  is  responsible  for  carrying  out  the  agency’s  mission 
assurance  strategy  (Figure  B-2).  QS  is  a  standalone  organization  and  reports 
directly  to  the  MDA  Director  on  matters  relating  to  Quality,  Safety,  and  Mission 
Assurance  (QSMA).  According  to  the  QS  Concept  of  Operations,  "QS  functions  as 
an  independent,  unfettered  and  unrestricted,  non-advocate  technical  organization 
for  MDA  with  a  specific  focus  on  mission  success  and  personnel  safety."^ 


^  QS-SOP-Ol,  "Quality,  Safety,  and  Mission  Assurance  Directorate  Concept  of  Operations,  May  9,  2013,  Page  4 
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QS  executes  this  function  through  seven  groups,  which  are  BMDS  Assurance 
Integration  (QSI),  BMDS  Safety  (QSS],  BMDS  Safety  Officers  (QSC),  Safety  and 
Occupational  Health  (QSH),  Mission  Assurance  (QSA],  BMDS  Quality  (QSQ),  and 
Parts,  Materials,  and  Processes  (QSP)  as  seen  in  the  QS  organizational  chart 
(Figure  B-3].  Although  there  is  a  specific  mission  assurance  group  within  QS,  all 
of  the  groups  perform  some  mission  assurance  activities  outlined  in  the  MAG  and 
MDA  Assurance  Provissions  [MAP]. 


Figure  B-3.  Quality,  Safety,  and  Mission  Assurance  (QSJ  Directorate 
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The  main  functional  groups  that  support  mission  assurance  are  QSQ  QSA,  QSP, 
QSI  and  QS  program  support  personnel. 

QSQ  ensures  that  the  quality  assurance  requirements  are  enforced  and  incorporated 
into  MDA  contracts  by  incentivizing  suppliers  to  provide  quality  products. 
QSQ  also  maintains  the  supplier  road  maps  (SRM),  which  documents  each 
program’s  supplier  down  to  the  fourth  tier.  In  addition,  they  support  MDA  test 
programs  by  reviewing  and  providing  input  to  test  event  certification  plans, 
certification  data  plans,  and  verifies  test  configurations. 
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QSA  ensures  that  the  mission  assurance  requirements  in  the  MAP  are  enforced. 
They  do  this  through  participation  in  design  reviews,  design  certification  reviews, 
manufacturing  readiness  assessments,  manufacturing  process  analysis,  pedigree 
reviews,  and  flight  and  ground  tests  participation.  They  also  work  with  program 
personnel  to  help  prepare  and  identify  program  risks  and  mitigation  plans. 

QSP  ensures  use  of  authentic,  quality,  and  reliable  parts  and  materials.  QSP 
maintains  and  enforces  the  MDA  Parts,  Materials,  and  Processes  Mission  Assurance 
Plan  (PMAP],  which  defines  Parts,  Materials,  and  Processes  (PMP]  requirements 
for  all  new  or  modified  safety  and  mission  critical  products  and  systems  developed 
for  MDA.  In  addition,  they  maintain  and  enforce  the  MDA  policy  on  purchasing 
electronic  parts  to  address  counterfeiting.  Lastly,  QSP  is  leading  a  team  to  develop 
an  MDA  corrosion  prevention  program. 

QSl  consists  of  personnel,  known  as  MDA  Assurance  Representatives  [MARs], 
who  are  permanently  located  at  MDA  contractor  facilities  that  produce/integrate 
MDA  critical  assets.  MARs  are  also  located  at  Vandenberg  Air  Force  Base  and 
Ft.  Greely  Alaska  launch  sites.  QSl  currently  has  26  MARs  that  are  stationed  at 
20  locations  across  the  U.S.  overseeing  the  day-to-day  operations.  In  addition  to 
covering  their  primary  facilities,  MARs  also  evaluate  other  suppliers  within  the  MDA 
supply  chain.  Their  boots-on-the-ground  presence  allows  for  continuous  process 
improvement,  implementation  of  industry  best  practices,  technical  oversight  of 
the  supply  chain,  and  formalized  facility  assessments.  Facility  assessments  cover 
electrical,  electronic,  electromechanical  (FEE)  parts,  software,  design  and 
workmanship,  work  instructions,  manufacturing  and  tooling,  cleanrooms,  electrostatic 
discharge,  foreign  object  debris  ,  safety,  training,  and  operator  certification,  critical 
lifts  and  moves.  Material  Review  Boards,  configuration  management  processes 
and  metrology  MARs  produce  reports  highlighting  hardware/software  risks  and 
facility  areas  for  improvement.  QSl  personnel  work  with  local  Defense  Contract 
Management  Agency  representatives  to  create  surveillance  requirements  and  ensure 
quality  products  are  produced  by  the  supplier  per  contract  requirements.  Lastly, 
QSl  personnel  lead  and  participate  in  formal  quality  and  mission  assurance 
audits  throughout  the  supply  chain. 
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There  are  167  QS  personnel  embedded  in  MDA  program  offices  to  ensure 
that  QSMA  requirements  are  met.  They  work  as  program  personnel  to  ensure 
requirements  are  included  in  contracts  awarded  to  MDA  suppliers  and  that  award 
fee  criteria  incorporate  QSMA  provisions.  In  terms  of  program  execution,  QS 
program  personnel  are  responsible  for  contract  requirements  reviews,  design 
reviews,  ground  and  flight  tests,  manufacturing  readiness  review,  first  article 
inspections,  hardware  acceptance  reviews,  pedigree  reviews,  and  failure  review 
boards/failure  investigations. 

Observation  1 

MDA  has  two  main  documents,  which  identify  the  standards,  requirements,  and 
engineering  principles  that  are  applied  to  each  program,  acquisition,  and  contract. 
This  ensures  that  MDA  programs  and  contracts  execute  to  the  same  baseline 
mission  assurance  standards  and  meet  a  minimum  level  of  quality. 

The  two  main  documents,  the  MAP  and  the  PMAP,  are  supported  by  additional 
internal  standards,  policies,  and  processes.  Flow  down  of  mission  assurance 

standards,  requirements,  and  principles  are  ensured  through  contract  incentives. 

MDA  Assurance  Provisions  (MAP) 

QS  developed  the  MAP,  which  establishes  quality,  safety,  and  mission  assurance 
processes  and  disciplines  required  throughout  the  acquisition  process  for  each 
program  contract.  They  developed  the  MAP  by  taking  standards  and  requirements 
applicable  to  the  mission  of  the  agency  from  industry  best  practice  such  as 

ANSI/EIA-632-1998  Process  for  Engineering  a  System  and  IEEE  Standard 

1012-1998  Software  Verification  and  Validation.  MAP  also  aligns  with  the  MAG 
in  several  areas  including  technical  and  mission  assurance  reviews.  Furthermore, 
it  provides  MDA  with  methods  to  measure,  verify,  and  validate  mission  success 
through  the  collection  of  metrics,  risk  assessment,  technical  evaluations,  and 
independent  assessments  and  reviews. 

MDA  deputates  are  responsible  for  developing  BMDS  subsystems  such  as 

Terminal  High  Altitude  Air  Defense  and  Ground-Based  Midcourse  Defense  are 
required  to  develop  a  Mission  Assurance  Implementation  Plan  (MAIP)  to  describe 
how  the  MAP  is  implemented  on  their  programs.  MDA  contracts  incorporate 
QSMA  requirements  to  promote  flow  down  of  requirements  and  best  practices 
from  prime  contractors  down  to  the  lower-tier  subcontractors.  For  example,  the 
Ground  Based  Missile  Defense  Development  and  Sustainment  Contract  include 


DODIG-2014-116  I  17 


Appendixes 


criteria  and  metrics  for  non-conformances,  unverified  failures,  quality  escapes, 
sibling  risks,  repeat  nonconformance,  first  pass  yield,  and  cost  of  rework,  repair, 
scrap  or  use  as  is. 

Parts,  Materials  and  Processes  Mission  Assurance  Plan  (PMAP) 

QS  established  the  PMAP,  which  identifies  requirements  for  selection,  approval, 
and  overall  management  of  PMP  used  in  MDA  products  and  systems.  This  plan 
documents  a  coordinated  approach  needed  to  maintain  the  highest  quality, 
reliability,  and  availability  of  MDA  products  and  systems  by  using  part  review 
boards  at  the  program  and  agency  level.  The  PMAP  is  implemented  in  all  MDA 
mission  and  safety-critical  hardware  contracts.  The  PMAP  requires  suppliers  to 
purchase  parts  from  authorized  sources,  or  perform  appropriate  testing  of  parts 
from  unauthorized  sources  to  mitigate  the  potential  risks  that  counterfeit  parts 
may  infiltrate  the  BMDS. 

Similar  to  the  MAP,  all  MDA  programs  are  required  to  develop  a  program-level 
PMP  plan,  which  identifies  the  level  of  PMAP  compliance  that  their  contractors  are 
required  to  meet.  PMP  activities  within  each  program  are  coordinated  with  their 
respective  PMP  Control  Board  (PMPCB].  There  is  an  agency  level  PMP  board 
that  handles  system-level  PMP  activities  and  issues.  QS  staffs  a  PMP  Advisory 
Group  (PMAG],  which  is  a  part  of  the  PMPB  and  supports  each  Program  PMPCB 
as  required. 

Observation  2 

MDA  employs  and  incorporates  several  types  of  independent  mission  assurance 
and  quality  reviews  throughout  the  product  lifecycle.  These  reviews  help  ensure 
that  mission  assurance  policies,  standards,  and  contract  requirements  are 
being  implemented. 

The  MDA  Director  delegated  authority  to  QS  to  institute  an  audit  program  to 
validate  if  hardware  and  software  products  are  acceptable.  The  program  validates 
products  against  engineering  design  requirements,  compares  qualification  and 
acceptance  test  methods  against  MDA  and  industry  standards,  validates  end-item 
flight  readiness,  and  examines  supplier  QSMA  practices  and  procedures.  The  MDA 
Audit  Program  consists  of  Mission  Assurance  Audits,  Mission  Focused  Audits, 
and  Facility  Checklist  Assessments.  Procedures  for  each  of  these  assessments  are 
documented  in  the  MDA  QSMA  Audit  Program  Standard  Operating  Procedure. 
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QS  functional  and  program  personnel  conduct  audits  and  assessments  at  MDA 
supplier  facilities.  These  audits  are  internally  scheduled  in  advance  but  are 
considered  "no-knock"  audits  to  the  supplier  or  contractor.  According  to  QS, 
"no-knock"  audits  provide  an  opportunity  for  accurate  assessments  and  insight  into 
a  supplier’s  actual  operating  environment.  Once  onsite,  the  audit  team  evaluates 
the  supplier’s  adherence  to  existing  contract  requirements,  internal  procedures, 
and  MDA  and/or  industry  best  practices.  These  audits  and  assessments  also 
determine  the  effectiveness  of  the  QSMA  strategies  and  processes  of  MDA  suppliers. 

MDA’s  audit  program  has  uncovered  several  significant  findings.  For  example,  an 
MDA  contractor  purchased  FEE  parts  for  a  rocket  motor  controller  from  an 
unauthorized  supplier,  increasing  the  risk  that  counterfeit  parts  were  used.  The 
purchased  EEE  parts  were  not  subjected  to  standard  authenticity  testing  at  time 
of  purchase.  As  a  result  of  this  finding,  the  contractor  took  corrective  actions  to 
prevent  further  occurrences  of  counterfeit  parts. 

Conclusion 

MDA  accomplishes  its  mission  assurance  strategy  by  designating  QS  as  an 
independent  technical  organization  with  a  focus  on  quality  and  mission  success. 
QS  performs  technical  assessments,  provides  recommendations  for  risk  mitigation 
and  acceptance,  and  provides  mission  readiness  statements  at  critical  readiness 
reviews,  and  facilitates  supplier  development  to  improve  site/supplier  mission 
safety  and  reliability.  These  processes  and  tools  such  as  the  MAP,  PMAP,  and  its 
audit  program  supports  MDA’s  goal  of  ensuring  mission  success. 

Air  Force  -  Space  and  Missile  System  Center 

Background 

The  Space  and  Missile  Systems  Center  (SMC],  located  at  Los  Angeles  Air  Force 
Base  in  El  Segundo,  CA,  is  a  subordinate  unit  of  the  Air  Force  Space  Command 
at  Peterson  Air  Force  Base,  CO.  It  is  the  center  of  excellence  for  acquisition  of 
military  space  systems.  SMC  conducts  research,  development,  procurement, 
deployment  and  sustainment  of  various  space  systems.  It  supports  this 
mission  with  an  average  budget  of  $8.66  billion  as  calculated  from  FY2009 
through  FY2013. 
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SMC  equips  U.S.  and  allied  forces  with  satellites,  command  and  control  systems, 
and  launch  systems  in  support  of  global  military  operations.  SMC  programs  focus 
on  space  force  enhancements  including  communications,  navigation,  tracking 
satellites,  space  support  to  include  launch  systems,  satellite  control  networks, 
and  force  application.  SMC  develops,  acquires,  fields  and  sustains  systems  in  four 
major  mission  areas.  These  areas  are: 

•  Space  superiority,  which  includes  programs  such  as  Space  Based 
Surveillance  constellations  of  satellites  and  the  Space  Fence; 

•  Space  support,  which  includes  launch  systems,  range  support,  and 
satellite  networks; 

•  Space  force  enhancement,  which  includes  programs  such  as  Military 
Satellite  Communications  Systems,  Global  Positioning  Systems,  Space 
Based  Infrared  Systems,  and  nuclear  detection;  and 

•  Force  application,  which  supports  conventional  missiles  and  prompt 
global  strike. 

SMC  Mission  Assurance  Approach 

SMC  approach  to  mission  assurance  is  through  the  development  and  use  of 
technical  specifications  and  standards  as  an  element  of  acquisition  practices  and 
the  use  of  independent  assessments  and  technical  reviews  of  programs.  SMC 
developed  its  approach  to  mission  assurance  after  a  string  of  launch  failures, 
which  were  attributed  to  relaxed  requirements  because  of  acquisition  reform, 
which  occurred  in  the  late  1980s  and  early  1990s.  SMC  implemented  several 
initiatives  to  improve  the  probability  of  mission  success  through  their  back-to-basics 
approach  implemented  in  the  2000s.  The  approach  focused  first  on  launch  process 
revitalization  then  expanded  into  a  larger  systems  engineering  revitalization 
campaign  across  the  organization.  This  back-to-basics  approach  focused  on 
processes  and  procedures  to  bring  back  key  specifications  and  standards.  This 
involved  industry  partnerships  and  collaboration  with  other  civil  agencies  to  share 
lessons  learned  and  best  practices  across  the  space  community.  These  processes 
included  using  the  MAG  and  space  flight  worthiness  criteria,  which  provided  a 
standard  to  assess  safety,  suitability,  reliability,  quality,  and  effectiveness.  They 
use  an  Independent  Readiness  Review  Team  (IRRT],  which  conducts  independent 
assessments  of  the  program.  The  IRRT  reviews  artifacts  such  as  pedigree  data  and 
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test  results  to  independently  identify  risk  using  mission  focus  areas  and  expert 
judgment.  It  also  provides  risk  assessment  and  recommendation  at  key  readiness 
milestones  such  as  space  and  launch  vehicle  ship  readiness,  and  launch  readiness. 
This  back-to-basics  approach  brought  back  many  of  the  principles  of  mission, 
product  and  quality  assurance  lost  during  acquisition  reform  allowing  SMC  to 
improve  its  overall  mission  success.  Overall,  the  SMC  mission  assurance  process 
ensures  safety,  suitability,  reliability,  quality,  and  effectiveness  of  the  program 
and  system. 

Observation  1 

SMC  uses  an  independent  engineering  directorate  within  its  organizational 
structure  to  ensure  mission  assurance  is  incorporated  into  programs  early  in 
the  acquisition  lifecycle  while  continuously  providing  systems  engineering  support 
to  SMC  programs  throughout  the  lifecycle.  This  independence  from  the  program 
offices  ensures  that  quality  and  mission  assurance  practices  are  not  inadvertently 
compromised  in  the  pursuit  of  cost  and  schedule  efficiencies. 

The  SMC  Chief  Engineer  is  responsible  for  ensuring  center-wide  application, 
implementation  and  adherence  to  all  policies  and  best  practices.  The  Chief  Engineer 
does  this  through  the  SMC  Engineering  Directorate  (SMC/EN].  SMC/EN  provides 
independent  assessment  and  analysis  of  programs  in  support  of  the  Program 
Executive  Office  (PEO)  for  Space.  It  also  provides  technical  assistance  to  program 
offices  through  their  engineering  Cadre  team  program.  The  Cadre  team  consists 
of  Government  and  contractor  subject  matter  experts  [SME)  in  the  systems 
engineering  directorate  that  provide  daily  technical  assistance  to  program  offices. 
The  team  interacts  directly  with  the  engineers  and  program  managers  to  provide 
technical  advisory  services  in  addition  to  provide  independent  assessments 
to  programs.  Its  ultimate  goal  is  to  prevent  the  reduction  of  mission,  quality,  and 
product  assurance  by  the  program  manager  in  the  pursuit  of  cost  and  schedule. 
Similarly,  the  Engineering  Directorate  team  conducts  independent  reviews  on  the 
program,  and  provides  recommendation  to  the  program,  manager  and  to  the  PEO 
for  Space.  The  team  reviews  and  approves  contract  requirements,  acceptance  of 
all  decision  and  launch  readiness  reviews,  and  assesses  program  offices  to  ensure 
readiness  to  enter  operational  testing. 
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Observation  2 

SMC  uses  a  set  of  69  standards  and  policies  to  attain  and  ensure  mission  success, 
which  every  SMC  program  manager  must  consider  and  apply  to  their  programs. 
These  baseline  standards  and  policies  includes  at  a  minimum  mission  assurance, 
quality  and  safety  principles  and  processes;  and  assurance  disciplines  providing  the 
program  manager  a  stable  starting  point  to  ensure  mission  success.  Additionally, 
they  provide  a  baseline  for  analysis  and  assessments  of  the  program  and  the 
contractor’s  technical  performance. 

Of  the  69  standards,  two  standards  that  stand  are  SMC-S-001,  "Systems 
Engineering  Requirements  and  Products,"  and  SMC-S-019,  "Program  and 
Subcontractor  Management."  They  help  the  program  manager  ensure  that  the 
proper  engineering  and  assurance  standards  and  processes  are  included  on  the 
program  and  in  contracts.  SMC-S-001,  defines  the  Government’s  requirement  for 
a  disciplined  systems  engineering  approach  to  systems  engineering.  It  specifies 
the  government’s  requirements  for  executable  contractor  systems  engineering 
efforts  and  can  be  used  as  a  guide  by  the  tasking  activity  to  assist  in  systems 
engineering  planning  and  management.  While  SMC-S-019  establishes  the 
requirements  for  the  program  and  subcontractor  management  program  to  ensure 
that  all  process,  roles,  responsibilities,  and  resources  affecting  the  control  of  the 
program  are  defined. 

SMC/EN  highly  recommends  that  mission  assurance  standards,  policies  and 
principles  be  placed  in  the  initial  request  for  a  proposal  and  awarded  contract.  Any 
deviation  from  standards,  policies  and  principles  must  be  formally  approved  by 
SMC/EN  and  must  meet  the  intent  of  the  standards.  This  ensures  that  mission 
assurance  best  practices  are  applied  to  weapon  systems  acquisition  throughout 
the  lifecycle  of  the  program.  The  MAG  principles  are  then  flowed  down  to  the 
subcontractors  and  suppliers  through  the  prime  contract.  SMC/EN  ensures 
the  contractor  is  meeting  the  proper  standards  and  policies  through  technical 
reviews  and  audits  such  as  the  System  Requirement  Review,  System  Functional 
Review,  Preliminary  and  Critical  Design  Reviews,  Functional  and  Physical 
Configuration  Audits,  and  independent  readiness  reviews.  They  also  have  in-plant 
representatives  where  necessary  to  ensure  adherence  standards  and  requirements. 
Similarly,  SMC  contracts  ensure  proper  systems  engineering  rigor  and  disciplines 
are  reflected  in  the  acquisition  strategy,  request  for  proposal,  and  contract. 
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Conclusion 

SMC  mission  assurance  process  assesses  and  ensures  safety,  suitability,  reliability, 
quality,  and  effectiveness  of  the  program  and  system.  SMC  does  this  through 
SMC/EN,  which  provides  independent  engineering  support  to  the  chief  engineer 
and  program  managers  through  the  issuance  of  baseline  standards,  policies  and 
requirements,  and  technical  assessments  and  program  reviews.  SMC  mission 
assurance  approach  ensures  the  proper  standards,  polices,  and  requirements  are 
on  contract  at  the  start  of  the  program  and  throughout  the  lifecycle.  Furthermore, 
they  institute  several  technical  reviews  and  audits  within  their  processes  to 
ensure  systems  have  met  requirements  before  proceeding  to  the  next  phase 
or  milestone. 

Navy  -  Strategic  Systems  Programs 

Background 

The  Strategic  Systems  Program  [SSP)  is  responsible  for  the  Trident  strategic 
weapon  system.  SSP  has  a  50-year  history  of  providing  credible  sea-based 
deterrent  missile  systems  and  numerous  successful  flight  tests.  It  is  the  Department 
of  the  Navy  organization  that  directs  the  end-to-end  effort  of  the  Navy’s  nuclear 
deterrent  Strategic  Weapon  System  to  include  system  acquisition,  training, 
equipment  sustainment,  and  facilities;  and  fulfill  the  terms  of  the  U.S.  and  UK 
Polaris  Sales  Agreement.  SSP  is  responsible  for  every  aspect  of  the  Strategic 
Weapons  System  (SWS)  from  concept,  design  and  development,  production, 
deployment,  protection,  and  operational  support;  through  system  retirement  and 
disposal.  They  have  an  average  budget  of  $2.57  billion  as  calculated  from  FY2009 
through  FY2013. 

SSP  is  a  vertical  hierarchy  organization  with  clear  lines  of  responsibility,  authority, 
and  accountability  and  is  aligned  to  the  SWS  subsystems.  The  organizational  chart 
in  Figure  B-4  shows  the  overarching  SSP  structure.  The  SSP  Director  has  overall 
accountability  and  three  division  level  direct  report  offices.  These  division  level 
direct  reports  are: 

•  Nuclear  Weapons  Safety  and  Security  Division  (SP30),  responsible 
for  coordinating  policies  associated  with  the  safety  and  security  of 
nuclear  weapons; 
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•  Technical  Division  (SP20),  accountable  for  the  technical  aspects  of 
the  weapon  system  including  design,  production,  maintenance  and 
operations;  and 


•  Plans  and  Programs  Division  (SPIO),  which  provides  supporting  program 
planning  functions  and  manages  resources  and  support  services. 

Figure  B-4.  SSP  Organizational  Structure 


Source:  Strategic  Systems  Programs 
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The  Technical  Division  is  responsible  for  mission  assurance  and  is  organized 
into  branches  aligned  with  SWS  subsystems  (Figure  B-5). 


Figure  B-5.  Technical  Division  (SP20)  Organization  Chart 


Source:  Strategic  Systems  Programs 

The  Technical  Division  sets  policies,  flows  down  requirements,  guides  technical 
management,  and  provides  oversight  to  ensure  product  assurance,  quality  assurance, 
and  SSP  success.  The  Chief  Engineer  and  his  staff  ensure  technical  disciplines, 
such  as  quality,  reliability,  maintainability,  and  product  assurance  are  included 
within  program  management  activities  conducted  at  headquarters,  field  activities, 
contractor  locations  and  other  support  activities.  They  also  support  and  ensure 
technical  communication  is  occurring  between  the  branches  and  divisions  of  SSP. 
SSP  executes  the  principles  of  mission  assurance  through  its  technical  management 
and  oversight  processes  by  using  proven  engineering  principles,  risk  management 
techniques,  and  independent  assessments  throughout  the  programs  lifecycle. 

Observation  1 

SSP  has  three  main  documents  that  identify  the  standards,  requirements,  and 
engineering  principles  that  are  applied  to  each  acquisition,  contract,  and  program. 
This  ensures  each  acquisition,  contract,  and  program  starts  with  the  same  baseline 
standards  and  meets  a  minimum  level  of  quality  and  product  assurance  criteria. 
The  three  main  documents  are  as  follows. 

•  Technical  Objectives  Guide  (TOG),  which  is  the  top-level  specification.  It 
guides  development;  identifies  systems  engineering  requirements;  and 
specifies  performance,  reliability  and  maintainability  requirements  for 
operations,  sustainment,  and  overall  test  methodology. 
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•  Strategic  Systems  Program  Organization  Manual  (SORM],  which  sets  the 
organizational  structures,  relationships,  and  functions  of  the  SSP.  The 
SORM  identifies  key  offices  for  technical  direction,  policies,  requirement 
flow  down,  technical  and  program  oversight,  manufacturing,  testing, 
and  independent  assessment. 

•  Technical  Program  Management  Requirements  for  Strategic  Systems 
Programs  Acquisitions  Document  [T9001B),  which  is  the  baseline 
contract  document  for  quality  and  product  assurance  and  specifies 
the  management  actions  and  technical  disciplines  to  be  invoked  on 
SSP  contracts. 

These  three  documents  are  supported  by  additional  internal  standards  and 
policies  to  facilitate  implementation,  process  flow,  and  overall  best  practices.  SSP 
identified  43  documents  that  address  technical  management,  oversight,  assessment, 
reporting,  issue  resolution,  configuration  control,  interface  management,  and 
testing.  The  technical  branches  are  responsible  for  executing  their  programs 
in  accordance  with  these  documents  and  are  assessed  against  them  by  the 
SSP  Chief  Engineer  and  his  staff.  Finally,  these  documents  guide  and  help  the 
SSP  chief  engineer  ensure  mission  success  of  SSP  programs  through  product  and 
quality  assurance  and  risk  management  principles. 

The  T9001B  lays  out  application  of  proven  scientific,  engineering,  quality, 
and  program  management  principles  towards  the  goal  of  achieving  mission 
success.  It  covers  all  phases  of  life  cycle  support,  beginning  with  development 
and  extending  through  production,  operational  support  and  eventual  disposal. 
T9001B  is  a  compendium  of  quality,  product,  safety,  and  guidelines  that  program 
managers  use  to  define  specific  contract  CDRLs.  It  calls  out  specific  design, 
reliability,  availability,  and  maintainability  requirements,  the  test  program 
approach,  configuration  management  program,  supplier  management  process,  and 
production  standards  to  be  included  on  the  contract.  Program  managers  can 
tailor  T9001B  based  on  specific  components,  lifecycle  phase,  and  contract  type. 
The  SSP  Chief  Engineer  Office  (in  particular  the  Engineering  Manager  Section]  is 
required  to  review  and  concur  with  the  tailoring.  This  ensures  a  disciplined 
approach  and  application  of  proven  scientific,  engineering,  and  quality  principles 
to  ensure  mission  success. 
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The  SSP  SORM  specifies  the  functions  of  the  SSP  organization  and  the  processes 
and  products  for  which  the  organization  is  responsible.  The  organizational 
functions,  in  particular,  those  of  the  SSP  Technical  Division,  require  the  application 
of  proven  scientific,  engineering,  quality,  and  program  management  principles. 
Three  examples  are  the  assigned  functions  of  the  Engineering  Manager  Section, 
Evaluations  and  Assessments  Section,  and  Missile  Branch.  The  Engineering 
Manager  Section  develops  policy  and  program  guidance  for  Technical  Program 
Management  (TPM],  Product  Assurance,  and  Quality  systems  for  the  SWS  during 
all  phases  of  the  SWS  life  cycle  at  contractor  facilities.  Government  shore  facilities, 
and  in  the  operational  Naval  Fleet.  The  Evaluation  and  Assessment  Section 
provides  SSP  technical  program  management,  evaluations,  audits,  and  management 
reviews.  The  Missile  Branch  executes  planning,  budgeting,  directing,  and  technical 
management  of  programs  to  research,  design,  develop,  test,  qualify,  and  install 
the  missile  system,  and  related  support  equipment. 

In  summary,  the  three  main  documents  guide  SSP  system  acquisition  and  program 
execution.  These  internal  policies,  procedures,  and  standards  are  applied  to  system 
acquisition  to  specify  system  and  technical  requirements  that  drive  quality  and 
product  assurance.  The  documents  direct  responsibility,  management,  and  technical 
performance  of  the  SSP  organization  to  guide  technical  program  management 
and  engineering. 

Observation  2 

Evaluation  and  reviews  are  conducted  on  SSP  prime  contractors,  major 
subcontractors,  and  SSP  Government  Field  Activities  and  Program  Management 
Offices.  SSP  employs  and  incorporates  several  types  of  independent  technical 
and  quality  reviews,  and  evaluations  into  their  processes.  These  reviews  help 
ensure  that  policies,  standards,  and  contract  requirements  are  being  implemented, 
which  results  in  a  higher  probability  of  mission  success. 

There  are  five  primary  independent  technical  and  quality  reviews  and  evaluations 
that  ensure  program  success:  Technical  Program  Management  Evaluation  [TPME], 
Management  Review  (MR),  Facility  Technical  Proficiency  Evaluation  (FTPE), 
Demonstration  and  Shakedown  Operation  (DASO),  and  Strategic  Systems  Program 
Alteration  (SPALT)  Program.  The  teams  that  execute  TPMEs,  MRs,  FTPEs,  and 
DASOs  are  independent  of  the  unit  under  review  or  evaluation.  The  SPALT  program 
initiates  with  a  pre-proposal  for  change  or  alteration  to  a  SWS  or  AWS,  which  if 
approved  by  the  responsible  SSP  field  office  is  submitted  as  a  proposal  for  review 
and  evaluation  by  Naval  activities  and  approval  by  the  SSP  Technical  Director. 
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TPMEs  assess  contract  compliance  and  are  typically  performed  every  3  years. 
The  TPME  evaluates  the  onsite  Program  Management  Office  and  associated 
contractor  performance  to  ensure  it  is  meeting  contract  requirements  by  reviewing 
the  technical  specifications  and  manufacturing  processes  against  the  statement  of 
work  and  the  tailored  T9001B.  An  external  group,  called  the  Evaluation  and 
Assessment  Team,  made  up  of  product  assurance  subject  matter  experts  from 
NSWC  Corona  performs  the  TPMEs.  TPMEs  take  a  week  to  complete.  The  Chief 
Engineer  then  uses  the  information  from  the  TPME  to  identify  underlying  issues 
of  the  non-compliance  with  requirements  and  subsequent  root  cause  analysis  and 
corrective  action  determination  are  conducted  for  each  issue. 

MRs  are  scheduled  every  3  years  to  evaluate  the  Government’s  performance  and 
implementation  of  programmatic  and  technical  functions  responsibilities.  The 
MR  ensures  effective  onsite  monitoring  and  technical  management  of  contractors 
by  the  respective  SSP  Program  Management  Offices.  Management  reviews  are 
conducted  on  Flight  and  Shipboard  Systems  Program  Management  Offices  (PMOs). 
The  SSP  Evaluation  and  Assessment  Section  chairs  the  evaluations  with  support 
from  NSWC,  Corona  and  SSP  Technical  Branch(es]. 

The  FTPE,  performed  every  3  years,  is  an  objective  evaluation  of  facility 
performance  to  assure  proper  accomplishment  of  the  SSP  mission.  SSP 
Headquarters,  with  support  from  SSP  PMOs,  NSWC  Corona,  other  SSP  field 
activities,  and  contractors  conduct  FTPEs  every  3  years  of  Strategic  Weapons 
Facility  Atlantic,  Strategic  Weapons  Facility  Pacific,  and  the  Naval  Ordnance  Test 
Unit.  The  FTPE  evaluates  both  the  Government  and  contractor  components 
to  properly  assess  operational  performance  and  systems,  the  potential  for 
performance  problems,  requirements  adequacy  and  validity,  and  the  need  for 
continuous  improvement  in  requirements,  systems,  and  procedures. 

DASOs  provide  assurance  that  ships  are  ready  to  carry  out  their  primary  mission. 
The  purpose  of  DASO  is  to  certify  weapon  system,  crew,  documentation,  and 
logistical  support  for  strategic  deployment  after  new  construction  or  overhaul. 
The  DASO  demonstrates  successful  firing  of  a  Trident  II  D5  Missile  and  validates 
from  end-to-end  that  the  SWS  meets  all  deterrent  mission  performance 
requirements.  The  SSP  Operations,  Evaluation,  and  Training  Branch  is  the  lead 
for  planning,  coordination,  technical  direction,  analysis,  execution,  and  conduct  of 
the  DASO  Program. 
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SSP  uses  the  SPALT  Program  for  configuration  control  and  configuration  status 
accounting  for  SWS  and  Attack  Weapon  System  (AWS]  Hardware  Configuration 
Items  and  Computer  Software  Configuration  Items.  The  SPALT  program  lays  out 
SSP’s  process  for  proposal,  evaluation,  approval,  implementation,  and  configuration 
management  of  changes  to  hardware  configuration  items  and  Computer  Software 
Configuration  Items  that  are  part  of  the  SWS  and  AWS.  The  SPALT  program 
ensures  that  changes  to  the  SWS  are  needed  and  provide  a  positive  advantage  to 
the  overall  system  program  considering  total  impact  on  cost,  personnel,  safety, 
and  system  effectiveness.  The  SPALT  Program  provides  the  policies,  controls, 
and  procedures  for  configuration  control  and  status  accounting  of  SWS  and  AWS 
hardware  and  software  items. 

Although  internal  policies,  procedures,  and  standards  specify  system,  quality,  and 
product  assurance  requirements,  SSP  uses  independent  assessments,  reviews,  and 
evaluations  to  ensure  compliance  to  requirements.  The  independent  assessments, 
reviews,  and  evaluations  are  applied  to  contractors  and  the  SSP  organization. 
Independent  assessments,  reviews,  and  evaluations  ensure  that  contractors  meet 
contract  requirements;  SSP  facilities  accomplish  their  assigned  mission;  SSP 
PMOs  provide  effective  oversight  of  contractors;  ships  are  prepared  to  execute 
their  missions;  and  SSP  executes  configuration  management  of  changes  and 
alterations  to  the  SWS  and  AWS. 

Conclusion 

SSP  executes  the  principles  of  mission  assurance  as  an  integral  part  of  technical 
program  management.  Its  internal  process,  procedures,  and  policies  ensure 
adherence  to  technical  management  and  systems  engineering  practices.  The 
execution  of  independent  assessments  and  certifications  ensure  compliance  and 
readiness  are  key  factors  SSP’s  success.  Internal  policy,  procedures,  and  standards 
direct  quality  assurance,  product  assurance,  and  mission  success  throughout 
the  system,  while  operational  exercises  provide  certification  that  ships  are 
mission  ready. 
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Principal  Deputy  for  the  Assistant  Secretary  of  Defense 
for  Research  &  Engineering 


RESEARCH 
and  EHGIiNEERING 


ASSISTANT  SECRETARY  OF  DEFENSE 

3030  D£rEhtSE  PENTAGON 
WASHINGTON,  DC  20301*3030 


SEP  1  20W 


MtMOfCWDUM  FOR  DEPUTY  INSPECTOR  GENERAL  POLICY  AND  OVERSIGHT 

THROUGH:  DIRTC  I'OR.  ACQUISITION  RESOURCES  AND  ANALYSIS i  i 

SUBJEC  T:  Response  to  DoDlG  Draft  Report  on  Assurance  Policy  Evaluation  Spacecraft  and 
Strategic  Systems  (Project  No,  D20l3-DT0TAD-0O(>2j 


As  requested,  I  am  providing  responses  to  the  general  content  and  recommendations 
contained  in  the  subject  report. 


Rmimmendation  Ar 

We  recommend  DASD(SE)  update  the  Defense  Acquisition  Guidebook  (DAG)  to  recommend 
that  Major  Defense  Acquisition  Programs  revievv,  tailor*  and  apply  applicable  mission  assurance 
concepts  and  principles,  such  as  those  found  in  the  Mission  Assurance  Guide,  when  developing 
Systems  Engineering  Plans  and  contract  requirements  to  promote  a  higher  probability  of  mission 
success. 


Resnunse: 

Concur.  DASD{SE)  will  update  the  DAG  Chapter  4  to  implement  the  DoDlG  recommendations. 
The  updated  DAG  will  be  released  in  2015  along  with  other  revisions  to  support  the  updated 
Department  of  r>cfensc  Instruction  5000.02, 


Recommetidatiun  B: 

We  recommend  DASDtSE)  review  the  best  practices  of  Missile  Defense  Agency  (MD.^h  Space 
and  Missile  Systems  Center  (SMC)  and  Strategic  Systems  Program  (SSP)  identifted  within  the 
report  and  incorporate  them  into  the  DAG,  Present  these  practices  at  the  next  DASD(SE)  bi¬ 
monthly  s>iilemK  cnyinecring  best  practice  meeting,  to  ensure  dissemination  of  these  best 
practices. 


Response: 

Concur.  The  principles  of  the  DAG  Chapter  4  are  consistent  with  the  DoDIG  recommendation 
and  DASD(SE)  will  continue  to  update  the  DAG  as  new^  practices  emerge.  For  example,  the 
DAG  will  be  updated  to  include  reference  to  standards  such  as  sv stems  engineering  standard 
IEEE  15288.L  the  technical  reviews  and  audits  standard  IEEE  1 5288.2,  and  the  configuration 
management  standard  SAE  FIA-649-1  once  these  standards  are  published  and  adopted  by  the 
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AWS 

Attack  Weapon  System 

BMDS 

Ballistic  Missile  Defense  System 

CDR 

Critical  Design  Reviews 

DAG 

Defense  Acquisition  Guidebook 

DASD(SE) 

Deputy  Assistant  Secretary  of  Defense  for  Systems  Engineering 

DASO 

Demonstration  and  Shakedown  Operation 

EEE 

Electrical,  Electronic,  Electromechanical 

ESOH 

Safety  and  Occupational  Health 

FASA 

Federal  Acquisition  Streamlining  Act 

FTPE 

Facility  Technical  Proficiency  Evaluation 

HSI 

Human  Systems  Integration 

lUID 

Item  Unique  Identification 

IRRT 

Independent  Readiness  Review  Team 

MAG 

Mission  Assurance  Guide  TOR-2007(8546)-6018 

MAIP 

Mission  Assurance  Implementation  Plan 

MAIS 

Major  Automated  Information  System 

MAP 

MDA  Assurance  Provissions 

MDA 

Missile  Defense  Agency 

MDAPs 

Major  Defense  Acquisition  Programs 

MDA/QS 

Quality,  Safety,  and  Mission  Assurance 

MARS 

MDA  Assurance  Representatives 

MR 

Management  Review 

ODASD(SE) 

Office  of  the  Deputy  Assistant  Secretary  of  Defense  for  Systems  Engineering 

PDR 

Preliminary  Design  Reviews 

PEG 

Program  Executive  Office 

PMAG 

PMP  Advisory  Group 

PMAP 

MDA  Parts,  Materials,  and  Processes  Mission  Assurance  Plan 

PMO 

Program  Management  Offices 

PMP 

Parts,  Materials,  and  Processes 

QSA 

Mission  Assurance 

QSH 

Safety  and  Occupational  Health 

QSI 

Assurance  Integration 

QSP 

Parts,  Materials,  and  Processes 
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QSQ 

QSS 

SDIO 

SEP 

SMC 

SSP 

SME 

SMC/EN 

SORM 

SPALT 

SRM 

SWS 

T9001B 

TOG 

TPM 

TPME 

USD(AT&L) 


BMDS  Quality 
BMDS  Safety 

Strategic  Defense  Initiative  Organization 
Systenns  Engineering  Plan 
Space  and  Missile  Systenns  Center 
Strategic  Systenns  Progrann 
Subject  Matter  Experts 

Space  and  Missile  Systenns  Center/Engineering  Directorate 
Strategic  Systenns  Progrann  Organization  Manual 
Strategic  Systenns  Progrann  Alteration 
Supplier  Road  Maps 
Strategic  Weapons  System 

Technical  Program  Management  Requirements  for  Strategic  Systems  Programs 
Acquisitions  Document 

Technical  Objectives  Guide 

Technical  Program  Management 

Technical  Program  Management  Evaluation 

Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
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U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

Monthly  Update 

dodigconnect-request@listserve.com 

Reports  Mailing  List 

dodig_report@listserve.com 

Twitter 

twitter.com/DoD_IG 

DoD  Hotline 

dodig.mil/hotline 
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